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Abstract 

■ - - _ In this paper, we show a connection between a certain online low-congestion routing 

■ problem and an online prediction of graph labeling. More specifically, we prove that if there 
«) I exists a routing scheme that guarantees a congestion of a on any edge, there exists an online 

■ prediction algorithm with mistake bound a times the cut size, which is the size of the cut 
induced by the label partitioning of graph vertices. With previous known bound of 0(log n) 

(jj '■ for a for the routing problem on trees with n vertices, we obtain an improved prediction 

I algorithm for graphs with high effective resistance. 

In contrast to previous approaches that move the graph problem into problems in vector 
space using graph Laplacian and rely on the analysis of the perceptron algorithm, our proof 
are purely combinatorial. Further more, our approach directly generalizes to the case where 
labels are not binary. 
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c/3 ! 1 Introduction 
o 

We are interested in an online prediction problem on graphs. Given a connected graph G = 

I {V,E) and a labeling i : V ^ { — 1,-|-1}, unknown to the prediction algorithm, in each round 

ly-^ ■ i, for i = 1,2, . . ., an adversary asks for a label of a vertex Vi G V, the prediction algorithm 

. provides the answer yi, and then receives the correct label jji = i{vi). The goal is to minimize 
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the number of rounds that the algorithm makes a mistake, i.e., rounds i such that yi 7^ yi. 
To make our presentation clean, in this work we do not count the mistake made on the first 



Q . question 

00 ! This problem has been studied with standard online learning tools such as the perceptron 

' algorithm. Herbster, Pontil, and Wainer |6j, and Herbster and Pontil [5j use pseudoinverse of 

graph Laplacian as a kernel and provide a mistake bound that depends on the size of the cut 
^ \ induced by the partition based on the real labeling of vertices and the largest effective resistance 

5h ' between any pair of vertices in the graph. Recently, Herbster [3] exploits the cluster structure 

of the labeling on the graph, and provides an improved mistake bounds. 

Pelckmans and Suykens [7] present a combinatorial algorithm for the problem that predicts 
a label of a given vertex based on known labels of its neighbors. They also prove a bound on 
the number of mistakes when the labels of adjacent vertices are known. However, their bound 
is very loose since it does not count every mistakes and their proof is still based on graph 
Laplacian. We shall compare the bound that we obtain with previous bounds of Herbster et. 
al. [Sl El [3] and of Pelckmans and Suykens [7] in Section 13.11 
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This work follows the initiation of Pelckmans and Suykens. We show connection between 
the prediction problem and the following online routing problem, first introduced by Awerbuch 
and Azar in their study of online multicast routing. Given a connected graph G = iy,E), 
the algorithm receives a sequence of requests ri,r2, . . ., where rj € V ^ and, for each rj, where 

1 > 0, has to route one unit of flow from to some previous know rj where j < i. The algorithm 
works in an online fashion, i.e., it has to return a route for rj before receiving other requests r^/, 
where i' > i. Given a set of routes, we define the congestion Cong{e) incurred on edge e € E, 
defined as the number of routes that use e. The performance of the algorithm is measured by 
the maximum congestion incurred on any edge. 

We prove, in Section [21 that if there exists an algorithm A with a guarantee that the 
congestion incurred on any edge will be no greater than a, there exists an online prediction 
algorithm with the mistake bound of 

a ■ \cut{i)\, 

where cut{£) be the set of edges joining pairs of vertices with different labels, i.e., cut{i) = 
{{u,v) eE:e{u) ^e{v)}. 

In Section [3l we apply the known congestion bound to show the mistake bound for the graph 
prediction problem, and compare the bound obtained with the bounds from previous results. 

We note that our approach directly generalizes to the case when labels are not binary (i.e., 
when the labeling function £ maps V to an arbitrary set L of labels) with the same mistake 
bound. 

2 Reduction to low-congestion routing 

We first present an online prediction algorithm from an online routing algorithm A. The pre- 
diction algorithm Pa is very simple, given a vertex Vi, it uses A to route one unit of flow from 
Vi to any vertices vj with known labels, it then returns the known label i{vj) as the prediction. 
We prove the following theorem. 

Theorem 1 If A guarantees that no edges is used more than a times, the prediction algorithm 
would make at most a ■ \cut{i)\ mistakes, not including the mistake made on the first query vi. 

Proof: We shall show that the number of mistake is at most a ■ \cut{£)\. Note that for each 
mistake Pa makes on vertex Vi, A routes Vi to some known vertex Vj along a path Pj. Since 
Pa predicts i{vj) and makes a mistake, we have £{vi) £{vj); thus. Pi must use some cut edge 
e in cut{t). We charge this mistake to e. We note that Pi may use many cut edges, but we 
only charge the mistake to one arbitrary edge. Since the routing produced by A uses each edge 
no more than a times, each cut edge is charged no more than a times as well. Therefore, the 
number of mistakes Pa makes must be at most a ■ \cut{£)\, as required. I 

We note that this proof does not use any fact that the labeling £ is binary; therefore, the 
proof holds for general labeling as well. 

3 Mistake bound 

To obtain the mistake bound, we first state the result on the online routing on trees. The 
theorem below first appeared in the work of Awerbuch and Azar [1], in which they called the 
problem restricted offline multicast, and has been discovered independently by Chalermsook 
and Fakcharoenphol |2j. We state the result in the form in [2j as it matches our settings. 



Theorem 2 (Theorem 4.4 in [Ij, Theorem 1 in [2j) For any tree T with n vertices and 
any sequence of vertices ti,t2, ■ ■ .t^ in T, there exists an efficient algorithm that finds a set 
of paths qi,q2, ■ ■ ■ ,Qk-i such that (1) qi connects ij+i to some tj, such that j < i, and (2) 
each edge in T belongs to at most O(logn) paths. Moreover the path qi depends only on paths 
qi,q2, . . .,qi-i. 

We note that the bound also holds for general graph G by taking T to be its spanning tree. 
Using Theorems [T] and [21 we obtain the following mistake bound. 

Theorem 3 For graph G = (F, E) and an unknown labeling I : V ^ L, there exists an efficient 
prediction algorithm that makes at most 

o(iog|F|) • \cut{e)\ 

mistakes, where cut{i) denotes the set of edges joining pairs of vertices with different labels. 

We note that for line graph, our algorithm is optimal. One can prove, in the same way 
as the proof of optimality of binary search, that an adversary can fool any algorithm to make 
r2(logn) mistakes on a line. 

3.1 Comparison to previous bounds 

We compare our mistake bound with the previous results. 

• Herbster et. al. [U [5] present an algorithm based on perceptron and prove the bound of 

4- \cut{l) \ ■Rg + 2, 

for the number of mistakes where Rq is the largest effective resistance between any pair 
of nodes in G (see [5], for the formal definition). We note that there are graphs where Rq 
is large, e.g, for line graph Rg = n — 1. Our bound is better when Rq = O(logn). 

While in the worst case Rq can be large, for many classes of graphs, e.g., highly connected 
graphs with small diameter, Rq can be very small. In [5|, they give an example where the 
cut size |cMt(£)| is linear, while Rg is 0{l/\cut{i)\). In this example, their mistake bound 
remains constant, while our bound grows with |cut(£)|. 

• In a recent paper, Herbster j4j exploits the cluster structures of graphs and proves the 
bound of 

Ar{G,p) + 4-\cut{i)\- p + l 

for any p > 0, on the number of mistakes, where A/'(X, p), the covering number, is the 
minimum number of sets of diameter p that contain all vertices of G under the semi-norm 
induced by the graph Laplacian (see [4] for definitions). 

This bound improves over previous bound in [S] when the graph has small number of 
clusters with small diameters. Herbster gives an example where the new algorithm makes 
only a constant number of mistakes while the algorithm from [5] makes linear mistakes. 
Again, in this example, our algorithm has linear mistake bound. 

We note that there is a trade-off between the diameter p of clusters and the number 
clusters in Herbster's bound. For many classes of graphs with large diameter, e.g. line 
graphs, using cluster structure does not help. The dependent on the cut size can still be 
i}{n) for graphs with n vertices. 



• Pelckmans and Suykens [7] present a simple combinatorial algorithm and show that the set 
M of vertices where the algorithm predicts incorrectly satisfies X^dgm '^m,v < 4 • \cut{i)\, 
where dM,v is the number of vertices adjacent to v that is also in M . Note that their 
bound only accounts for edges between two mistaken vertices. If there are no edges 
between vertices in M, their bound does not say anything. For example, consider the case 
with line graph with n vertices, where vertices 1,2,... ,n/2 have label +1 and vertices 
n/2 + 1, . . . ,n have label —1. The algorithm of Pelckmans and Suykens can make 
mistakes if an adversary asks the labels of 1, 3, 5, . . ., while the cut size is just 1. 

4 Open questions and discussions 

Our bound depends on the worst case bound on the congestion from the routing problem. 
However, the O(logn) bound seems very loose for dense graphs. It would be nice to see if one 
can find the connection between the worst case congestion and the effective resistance. We note 
that when the effective resistance is low, between any two nodes there must be many short 
disjoint paths, and this should help reducing the congestion. Also, there is extensive literature 
on online routing with small congestion (see, e.g., [HIEIIS]). Can these results be used to give 
better mistake bounds as well? 

We note that our proof cannot give a mistake bound smaller than \cut[£)\. To improve 
further, one need a way to account for cut edges that have not been charged. 

Finally, we wish to see any adversarial bound on the number of mistakes for an online label 
prediction algorithm. In this paper, we have shown that our algorithm is optimal (up to a 
constant factor) for line graphs. The ultimate goal would be to find an optimal algorithm for 
general graphs. 
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